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Abstract 

We develop the dynamic programming approach for a family of infinite horizon 
boundary control problems with linear state equation and convex cost. We prove 
that the value function of the problem is the unique regular solution of the associated 
stationary Hamilton-Jacobi-Bellman equation and use this to prove existence and 
uniqueness of feedback controls. The idea of studying this kind of problem comes 
from economic applications, in particular from models of optimal investment with 
vintage capital. Such family of problems has already been studied in the finite 
horizon case in |24j [25] . The infinite horizon case is more difficult to treat and it is 
more interesting from the point of view of economic applications, where what mainly 
matters is the behavior of optimal trajectories and controls in the long run. The 
study of infinite horizon is here performed through a nontrivial limiting procedure 
from the corresponding finite horizon problem. 

Keywords. Linear convex control, boundary control, Hamilton-Jacobi-Bellman equations, age- 
structured systems, optimal control, economic growth, vintage models. 
AMS (MOS) subject classification: 49J20, 49J27, 35B37. 

1 Introduction 

This paper is devoted to the study of a family of infinite horizon boundary control prob- 
lems with linear state equation and convex cost and of the associated stationary Hamilton- 
Jacobi-Bellman (briefly, HJB) equations by means of Dynamic Programming. More pre- 
cisely, we let H and U be separable real Hilbert spaces with scalar products {-\-)h and 
{■\-)u respectively, and we consider a dynamical system of the following type 




y'(r) = A y(r)+Bu(T), r e]t, +oo[ 
y(t) = xeH, 
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where H is the state space, y : [£,+oo[— > H is the trajectory, U is the control space 
and u : [t,+oo[— > U is the control, A : D(A ) C H — > H is the infinitesimal generator 
of a strongly continuous semigroup of linear operators {e Tj4 °} r >o on H, and the control 
operator B is linear and unbounded, say B : U — > [-D(Ag)]'. Besides, we consider an 
infinite horizon cost functional given by 

/+oo 
e -AT foo (y(r)) + /i Kr))]dr 

where the function go is convex and C l , and /io is l.s.c, convex, superlinear, possibly 
infinite valued, as better specified later. Our problem is that of minimizing Joo(t, x, u) with 
respect to u over a set U of admissible controls (which will be denoted with L p x (t, +00; U), 
that is an LP space with a suitable weight, as defined in Section [3]). 
Then the value function is defined as 

(1.3) Z OQ (t,x)= sup Joo(t,X,M). 

u£L p x (t,+oo;U) 

Since it is easily shown that Z^it.x) = e~ A *Z oo (0, x), it is enough to study the HJB 
equation associated to the problem with initial time t = 0, that is 

(1.4) - \^{x) + (ij/(x) I A x) H - hl(-B*i)\x)) + g(x) = 0, xeH 

whose candidate solution is ^00(0,0;). (Here and in the sequel, indicates the Legendre 
transform of the convex l.s.c. function Hq.) 

The problem has been already studied by Faggian and by Faggian and Gozzi in the 
papers [2H [25l [26l [27] in the case of finite horizon, with and without constraints on the 
control and on the state, yielding a definition of generalized solutions of the associated 
evolutionary HJB equation. This paper studies instead the infinite horizon case. 

It is well known that linear convex control problems with unbounded control operator B 
in Hilbert spaces arise when one rephrases in abstract terms some boundary control prob- 
lem for linear PDEs (or, more generally, problems with control on subdomains). Indeed, 
we motivate our framework with the application to a problem of optimal investment with 
vintage capital arising in economic theory (originally formulated by Barucci and Gozzi 
[Tlj [12] and then studied also in [28j [29J, [30]), that cannot be treated with the existing 
results and that we describe in detail in Section [5j We also observe that our framework 
adapts also to other optimal control problems driven by first order PDE's or by delay 
equations and arising in models of population dynamics (see e.g. [SI EE]), advertising (see 
e.g. [ZTJ ED ESI EH]), general equilibrium with vintage capital (see e.g. [HI l2"Tj). 

Our main results are stated in Section El in Theorems 13. 6[ |3~7T[ 13.81 where we prove that 
the value function Zq^O, •) is the unique regular (C 1 ) solution of the HJB equation (11.41) 
and that there exists a unique optimal control strategy in feedback form. Moreover the 
value function is the limit of value functions of suitable finite horizon problems. 

We obtain the results by means of the procedure introduced by Barbu and Da Prato 
[6] (see also Di Blasio [HI [20]) that consists, roughly speaking, in the following steps: 
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• Consider a family of suitable problems with finite horizon T, with value functions 

and show they are the unique regular solutions of the corresponding family of 
evolutionary HJB equations^ 

• Show that the value functions \I/t converge, as T — > +00, to a regular function \IZoc. 

• Prove that is the unique solution of the stationary HJB equation and that it 
is equal to the value function, Z^, of the infinite horizon problem with initial time 
t = 0; prove the existence and uniqueness of optimal feedbacks. 

In our (boundary control) case a sharp refinement of this methods is needed. Indeed, with 
respect to the papers quoted above, our problem features two new nontrivial difficulties: 

• The presence of the boundary control yields the unboundedness of the control op- 
erator B in the state equation (jl.lj) and, as a consequence, the discontinuity of the 
Hamiltonian in the HJB equation (11.41) . This fact, coupled with the non-analyticity 
of the semigroup generated by A, induces us to work in an enlarged space V D H . 
This setting was already introduced in j2U [25] to treat the corresponding finite 
horizon problem. Of course, since in the examples in Section [5] the parameters have 
significance only in H, we need to prove that when in the extended setting the ini- 
tial datum x is in H, then the whole optimal trajectory lies in H, and the optimal 
control behaves accordingly. 

• The running costs go and h Q are not bounded from below. This means that a two- 
sided inequality has to be proved in order to show the convergence as T — > +00. 
To this extent, we exploit the coercivity of the function ho to derive that optimal 
controls are bounded in L p x . 

We may say that the contribution of this paper is both to mathematics and to appli- 
cations: the results contained in the theorems in Section [3] extend the existing theory of 
regular solutions of HJB equations in Hilbert spaces to a new set of problems; on the 
other hand our results are the basis to start the study of the properties of the optimal 
state-control pairs in the economic problem, and in those other applications that can can 
be framed into the same setting. 

The paper is organized as follows. In next the rest of this Section (Subsection II. ip we 
review the main literature on HJB equations in Hilbert spaces. In Section [2] we recall 
the definition of strong solution and the results on existence and uniqueness of strong 
solutions in the finite horizon case, as they appear in [23]. In Section [3] we present the 
problem in the extended space and we state the main results. Section @] is devoted to 
proofs and Section to the description of the economic example. 

indeed these facts in our case were shown in [241 125] . refining the convex regularization method by 
Barbu and Da Prato contained in [6]. 
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1.1 The Literature 



We recall that optimal control problems for infinite dimensional systems and the associ- 
ated HJB equation have been studied in two different frameworks: one is that of classical 
and strong solutions, and the other is that of viscosity solutions. We recall also that, as 
far as we know, verification techniques have been performed in infinite dimension just in 
the classical/strong context, for they require the value function to be regular (at least in 
the state variable). 

Regarding Dynamic Programming for boundary control problems only few results are 
available. For the case of linear systems and quadratic costs (where HJB equation reduces 
to the operator Riccati equation) the reader is referred e.g. to the book by Lasiecka and 
Triggiani [37], to the book by Bensoussan, Da Prato, Delfour and Mitter [13], and, for 
the case of nonautonomous systems, to the papers by Acquistapace, Flandoli and Terreni 
[H El El H]- For the case of a linear system and a general convex cost, we mention 
the papers by Faggian [221 E31 EU ESI EE], by Faggian and Gozzi [27J. On Pontryagin 
maximum principle for boundary control problems we mention again the book by Barbu 
and Precupanu (Chapter 4 in [TO]). 

For the case of distributed control the literature is indeed richer: we refer the reader 
to Barbu and Da Prato [HI 0, E] for some linear convex problems, to Di Blasio [JjJl ED] 
for the case of constrained control, to Cannarsa and Di Blasio [15] for the case of state 
constraints, to Barbu, Da Prato and Popa [9] and to Gozzi [32l [331 134"] for semilinear 
systems. 

For viscosity solutions and HJB equations in infinite dimension we mention the series 
of papers by Crandall and Lions p2] where also some boundary control problem arises. 
Moreover, for boundary control we mention the papers by Cannarsa, Gozzi and Soner [16] 
and by Cannarsa and Tessitore [18] on existence and uniqueness of viscosity solutions of 
HJB equation. We note also that a verification theorem in the case of viscosity solutions 
has been proved in some finite dimensional case in [SHI EH] ■ 

Regarding applications, in addition to the economic literature recalled above, we refer 
the reader to the many examples contained in the books by Lasiecka and Triggiani [37] 
and by Bensoussan et al [13]. 

2 Preliminaries: the finite horizon case. 

We here recall all the relevant results on the finite horizon case that are needed in the 
sequel. According to the notation in [23], if X and Y are Banach spaces, we set 



Ltp(X;Y) 





sup 

x,y£X, x^y 




CUX) : 



{feC\X) 



If'L < 



+00} 



B P (X,Y): 



{f-x^m 



l/k : 




f<+oo}, B P {X):=B P (X,R). 
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Moreover we set 



E (X) := {w e B 2 {X) : w is convex, w G C Lip (X)} 

and, for T > 

y{[0,T] x X) = {w: [0,T] xX : w G C([0, T], B 2 (X)), 

w{t,-)eE (X), Wg[0,T], ^eC([0,T],B 1 (I,I'))} 

Then we consider two Hilbert spaces V, V, being dual spaces, which we do not identify for 
reasons which are recalled in Remark 12.21 and we indicate with (-, ■) the duality pairing. 
We set V as the state space of the problem, and denote with U the control space, being 
U another Hilbert space. 

Given an initial time t > an initial state x G V, a finite horizon T > t, a number 
p > 1, and a control u G L p (t, T; U) we consider the trajectory in V 

(2.1) y(r) = e {T ~ t)A x + J* e {r - a)A Bu(a)da, r G [t, T], 
and a profit functional of type 

(2.2) J T (t, x,u) = J [g (r, y{r)) + h (r, u(r))} dr + ^{y{T)). 

We deal with the problem of minimizing Jx(t,x,-) over all u G L p (t,T;U) taking the 
following set of assumptions on the data. 

Assumptions 2.1. 1. A : D(A) C V — > V is the infinitesimal generator of a strongly 
continuous semigroup {e rA } r >o on V'\ 

2. Be L(U,V); 

3. there exists Uq > such that \e rA x\ v , < e UJ ° T \x\ v ,, Vr > 0; 

4. g G 3>([0,T] x F'), t ^ [fe(t,")L e LH0,T) 

5. up g So(K'); 

6. /i(t, •) is convex, lower semi-continuous, d u h(t, •) is injective for all t G [0, T]. 

7. If is set H(t, u) := [/i(r, •)]*(«), then we assume ft G ^([0, T]xU), H(t, 0) = 0, and 
^Pte[o,T][Hu(t,-)} L < +oo. 

Remark 2.2. We do not identify V and V for in the applications the problem is naturally 
set in a Hilbert space H, such that V C H = H' C V (with all bounded inclusions). 
Indeed, in order to avoid the discontinuities due to the presence of B, as they appear in 
fll.ipfll.2p . we work in the extended state space V related to H in the following way: V is 
the Hilbert space D(A£) endowed with the scalar product (v\w)v '■= (v\w)h+(Aqv\Aqw)h, 
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V is the dual space of V endowed with the operator norm. Then assume that B G 
L(U, V), and extend the semigroup {e tA °} t >o on H to a semigroup {e tA } t > on the space 
V , having infinitesimal generator A, a proper extension of Aq. The reader is referred to 
[25] for a detailed treatment. I 

Remark 2.3. Note that the functions g and 4> arising from applications usually appear 
to be defined and C 1 on H, not on the larger space V. Then, we here need to assume 
that they can be extended to C l -regular functions on V - which is a non trivial issue. 
We refer the reader to Section \5\ to see how such extension is obtained in the specific case 
of the economic example, and to [21] and [25] for a thorough discussion of this issue. ■ 

Remark 2.4. In Assumption I2.1| 7]. we assumed Tl(t, 0) = 0. Such assmption is not 
restrictive since TH.it, 0) = — ini v€U h(t, v) and, if this value is not 0, we may reduce to this 
case simply setting g = g+mf v£U h(t, v) and h = h—inf v€U h(t, v) and treating the problem 
with g and h in place of g and h. Note also that the assumption dh(t, ■) injective is intended 
to yield a good definition for TC U as it is, roughly speaking, TC U = (dh)' 1 . Note also that 
once one has the datum h, its convex conjugate 7i is very often explicitly computed. Then 
the assumptions on H are essentially assumptions on its convex conjugate h, but more 
conveniently stated to ensure H has the desired properties. I 



Such optimal control problem can be associated by means of dynamic programming, to 
the following Hamilton- Jacobi-Bellman equation 



(2.3) 



v t (t, x) - H(t, -B*v x (t, x)) + (Ax\v x (t, x)) + g(t, x) = 0, (t, x) e [0, T] x V 
v(T,x) = <f(x), 



that can be written, by the change if variable v(t, x) = <fi(T — t, 
(2.4) 

'(fH(t, x) + U(T - t, -B*(j) x (t, x)) - {Ax, (f> x (t, x)) = g(T -t,x), (t, x) G [0, T] x V 
(f){0,x) = tp[x). 

Finally, the value function of the problem is defined as 

(2.5) W T (t,x)= inf J T (t,x,u), 

u£LP(t,T;U) 

Indeed in [21] Faggian proved existence and uniqueness of strong solutions, as defined 
shortly afterwards, for a class of more general HJB equations, that is 



(2.6) 



<f> t {t, x) + F{t, <f> x {t, x)) - (Ax, <f> x {t, x)) = g{T -t,x), {t, x) E [0, T] x V 
0(0, x) = (p(x), 
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where F satisfies 

(2.7) F E y([0, T] x V), F(t, 0) = 0, sup [F p {t, -)} L < +oo 

te[o,T] 

Note indeed that if we set 

F(t,p) := H{T-t,-B*p) = sup{(u| - B*p)u - h(T-t,u)}. 

then F satisfies (12 .7h and it is well defined for p in V, to which (p x (t,x) belongs. 

Definition 2.5. Let Assumptions \2.1\ 1—5, and {2.1) be satisfied. We say that 
4> E C([0,T],B2(V')) is a strong solution of < \2.6b if there exists a family {(p £ } e C 
C([0,T},B 2 (V')) such that: 

(i) <p £ (t,-) E C\ ip (V) and <f) £ (t,-) is convex for all t E [0,T\; <f) £ {0,x) = (p(x) for all 
x E V . 

(ii) there exist constants Ti,r2 > such that 

sup [<Pl(t)] L < r 1; sup \<f>%{t,Q)\v < T 2 , Ve > 0; 
te[o,T] te[o,T] 

(Hi) for all x E D(A), 1 1— > <f) e (t,x) is continuously differentiable; 
(i v ) <jf ^ ; as e -> 0+ ; m C([0, T], ^(V 7 )); 

(f) there exists g £ E C([0, T]; 62 (V 7 )) SlicA t/iat, /or a// 1 E [0,T] and s G -D(A), 
0f(t, *) - F(t, <f>l(t, x)) + (Ax, <b £ x (t, x)) = g £ (T - t, x) 
with g e (t, x) — > g(t, x), and J Q T \g e (s) — g(s)\c 2 ds — > 0, as e — » + . 
The main result contained in [23] is the following. 

Theorem 2.6. Let Assumptions \2. 1\ 1 — 5, and ( \2. ffi) be satisfied. There exists a unique 
strong solution (ft of ( 00 ) in the class C([0, T], B 2 (V')) with the following properties: 

(i) for all x E D(A), </)(-, x) is Lipschitz continuous; 

(ii) E y([0,T] x V'). Moreover the following estimate is satisfied for all t E [0, T] 

(2.8) [Ut)] L < e 2 ^y\ L + f e 2 ^- s \g x (T - s, -)} L ds. 

Jo 

Regarding applications to the optimal control problem, in |25j we were able to prove 
what follows. 

Theorem 2.7. Let Assumptions \2.1\ 1 — 7 be satisfied, and let <fi be the strong solution of 
(2.4) described in Theorem \2.6[ Then 

W T (t, x) = (f)(T -t,x), Vi E [0, T], Vx E V, 

that is, the value function Wt of the optimal control problem is the unique strong solution 
of the backward HJB equation (\2.3§ . 
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3 The infinite horizon problem 

We describe the abstract setup of the infinite horizon optimal control problem and we 
state the main result of the paper, namely Theorem 13.71 that establishes that the value 
function of our problem is the unique regular solution of the associated HJB equation. 
Some other important results follow, such as Theorem 13.81 on existence and uniqueness of 
optimal feedbacks, and Theorem I3.6[ establishing the connection between finite and infite 
horizon value functions. Proofs of all assertions are found in section HI 

We use the same framework as that in Section [2J for the finite horizon problem. As one 
expects, the state space is V and the control space is U. The state equation is given in 
V as 

(3.1) y{r) = e {T ~ t)A x + J e^ T ~ a)A Bu(a)da, r G [t, +oo[, 

while, for all x G V and t > 0, the target functional is of type 

/+oo 
e~ XT [go(y(r)) + h (u(r))}dT. 

We assume the following hypotheses: 

Assumptions 3.1. 1. A : D(A) C V — > V is the infinitesimal generator of a strongly 
continuous semigroup {e rA } r >o on V; 

2. Be L(U,V')\ 

3. there exists u G M such that \e rA x\ v , < e UJT \x\ v ,, Vr > 0; 

4. g ,4> eZ (V') 

5. h is convex, lower semi-continuous, d u h is injective. 

6. ^5(0) = 0, h* eZ (V); 

7. 3a > 0, 36 G R, 3p > 1 : h (u) > a\u\ p v + 6, Vn G U; 
Moreover, either 

8.a p > 2, A > {2uj Vw). 
or 

8.b A > uj, and ^0,00 G Bi(V). 

Remark 3.2. Note that the Assumption 13.11 [3] above implies that also Assumption 12.11 
[3] where ujq = uj V 0. 
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The functional Joo(£; x, u) has to be minimized with respect to u over the set of admis- 
sible controls 

(3.3) L p x (t, +00; U) = {ue Ll c (t, +00; U) ; 1 1-> u(t)e~^ e L p (t, +00; [/)}, 
which is Banach space with the norm 



\\ u \\Ll(t,+oc;U) = J \ u i T )\u e Xr dr= \\e p u\\ L P {t ,+oo;U) 
Similarly, the space L p x (t,s;U) endowed with the norm 



.MO 



A(-) 



\\u\\ L l(t,s;U) = J K r )l^e T dr=\\e * u\\ LP ^ s .u)- 

is a Banach space. Then (13. 3p is the natural set of admissible controls to get estimates in 
this setting (see e.g Lemma [4.51 and Lemma 14.71) . 
The value function is then defined as 

Zoo (t,x) = inf (t, x,u). 

u£L p x (t,+oo;U) 

As it is easy to check that 

Z oo {t,x) = e~ xt Z oo {0,x) 
one may associate to the problem the following stationary HJB equation 

(3.4) - hj,{x) + (V>'(x), Ax) - h*{-B*iP'(x)) + g(x) = 0, 

whose candidate solution is the function .^(O, ■). 

We will use the following definition of solution for equation (13.41) . 



Definition 3.3. A function if) is a classical solution of the stationary HJB equation l \3.4§ 
if it belongs to ^{V') and satisfies l \3.4\) pointwise for every x G D(A). 



Remark 3.4. The reader has certainly realized that Assumptions 13. ll [1 — 7] imply As- 
sumptions 12.11 [1 — 7]. Moreover, as mentioned thoroughly in Remark 12. 3^ we need to 
assume that the functions go and 0o can be extended to C 1 -regular functions on V. ■ 

Remark 3.5. See Remark 12.41 for some comments on ho and h* Q that apply also to this 
case. I 

Before proving that the value function of the infinite horizon problem starting at (0, x), 
namely Z oo (0,x), is the unique classical solution to the stationary HJB equation, some 
preliminary work is needed. First we show that (0, x) is the limit as t tends to +00 of a 
suitable family of value functions for finite horizon, along with their gradients. Doing so, 
we also establish that inherits from that family the C l regularity in x which we need 
to solve the stationary HJB equation, and which is so precious when building optimal 
feedback maps. 
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Theorem 3.6. Let Assumptions \3. 1] be satisfied. Let also <pT{t,x) be the unique strong 
solution to <\2.$ . Then the function 



is independent of T and there exists the following limit 

^ocix) := lim ty(t,x). 

The convergence is uniform on bounded subsets of V . Moreover, if A > wmax{2,p-}, 
then \IZoo G £ (V^). Moreover, for every fixed x G V 

^/ x (t,x) — > ^'^(x), weakly in V, as t — > +00. 

Hence, \IZoo being the candidate solution to the stationary HJB equation ( 13.41) . one 
shows what follows. 

Theorem 3.7. Let Assumptions \3.1\ hold. Then: 

(i) is the value function of the infinite horizon problem with initial time t = 0, that 

is 

^oo(x) = 2^(0, a;) = inf Joo(0, x,u). 

ugL^(0,+oo;l/) 

Moreover Z^it^x) = e~ A *\l/ 00 (x); 

(n) \lZoo is a classical solution (as defined in Definition ^. 3\) of the stationary Hamilton- 
Jacobi- Bellman equation (TO), that is 



-\*oo(x) + (V^x), Ax) - hK-B^x)) + g(x) = 0. 
(Hi) The function \lZoo is the unique classical solution to ( 3.4 ) ■ 



Once we have established that is the classical solution to the stationary HJB equa- 
tion, and that it is differentiable, we can build optimal feedbacks and prove the following 
theorem. 

Theorem 3.8. Let Assumptions \3.1\ hold. Let t > and x G V be fixed. Then there 
exists a unique optimal pair (u*,y*). The optimal state y* is the unique solution of the 
Closed Loop Equation 

(3.5) y(r) = e^ A x + ^ e^ A B(h* )> '(-B'V'Ms)))^, r G [t, +oo[. 

while the optimal control u* is given by the feedback formula 

u*{s) = (Kn-B^y^s))). 

4 Proofs of the main results 

In this section we prove the theorems stated in Section [3J 
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4.1 Auxiliary functions, equations and estimates 

We study infinite horizon by means of finite horizon. Then it is worth noting that, thanks 
to the particular dependence of data on the time variable, we can associate to the HJB 
equation arising in finite horizon the following equation: 



(4.1) 



Zt(t, x) - Xz(t, x) + (Ax, z x (t, x)) - h* (-B*z x (t, x)) + g (x) = 
z(T,x) = <f>o(x) 

and define a strong solution of (14.11) as follows. 

Definition 4.1. Let (t,x) G [0, T] x V . We say that Zt is a strong solution to fl^.ip if 

Z T (t,x) = e xt v T (t,x) 
with vt any strong solution to (ll?.5j) . in the sense of Definition \2. 5i 

Remark 4.2. Equation (14.11) is obtained formally from (12.31) with the change of variable 
v(t,x) = e~ xt z(t,x). Note that one could give a direct definition of solution of (14.11) 
(without passing through strong solutions of (12.31) ) in the spirit of Definition (12. 5p . I 

Note that 

ueL p (t,T;U) ^ ue L p x (t,T;U) 

so that all minimization procedure in Section 2 can be equivalently operated in L p (t, T; U) 
or in L p x (t,T;U). Then, recalling that the unique strong solution to (12. 3p is the value 
function of the optimal control problem (see [25]), the following result is readily proven. 

Proposition 4.3. Let Assumptions \2.1\ be satisfied, and let Zj- be the unique strong 
solution to CO). Then 



(4.2) Z T (t,x) = inf ( f e-^-% {y{r)) + h (u(r))}dr + e' x ^ <j> {y{T)) \ . 

We may also write a forward version of (14.11) . that is 

^ f (t, x) + Xip(t, x) - (Ax, ip x (t, x)) + h*(-B*^ x (t, x)) = g (x) 
ip(0,x) = (j>o(x) 



(4.3) 



with (t,x) G [0,T] x V and if)(t,x) = z(T — t,x), where Z is the unique strong solution 
of (14.11) . and then prove the following important result. 



Lemma 4.4. Let ^r(t, x) = Zt(T — t,x), where Z is given by Q^.ffi ). Then \1/ does not 
depend on T, that is 

(4.4) *(t,x) = * T (t,x)= inf ( fe-^igoiyir^ + hoiuiry^dT + e-^Myit)))- 
Hence, . Moreover a Dynamic Programming Principle holds 

(4.5) = iof (/ e- XT [go(y(r)) + h (u(r))]dT + e- Xs ^(t-s,y(s))), Vs G [0,t]. 
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Proof. By changing the variable, for all < s < t, and u G L p x (s, t; U), we have 
(4.6) J t (s, x, = e~ Xs J t _ s (0, x, u(- + s)). 

Then by definition of value function and (14. 2p . we have 

^(t, x)= inf J T (T — t,x,u) = inf J t (0,x, m) 

u£L p x (T-t,T;U) ueL p x (0,t;U) 

where the last equality is obtained by setting u(s) := u(s + T — t), and observing that 

ue L P X (T -t,T;U) <^=> ueL p x (0,t;U). 



Then f!4.4f) follows by generality of u. 

The proof of the Dynamic Programming Principle is standard and we omit it. 



Here follow some other technical results that will be frequently exploited in the sequel. 

Lemma 4.5. In Assumptions \3. 1\ if u G L p x (t,T; U) is an admissible control and y(r) = 
y(r;t,x,u) is the associated trajectory, and q = p/(p — 1), then for suitable positive 
constant C independent of t and x the following estimates hold: 



(4.7) 



e-^\u(T)\udr < 9(t, s)5 \\u\\ L p {ty 



(4.8) 



W)\ v > < Ce^[\x\ vl ^e{t,r)^u\\ L Pu T ,u)\ 



(4.9) 
with 



e- XT \y{r)\ vl dr<C 



\ X \ V ' + \\ U \\Ll{t,s;U) + 1 



Vs > t 



9(t,s) 



I A— pu>\ 

It- s 



tup 



A = up. 



Remark 4.6. Note that all inequalities in the following proof hold also for p = 2, but 
the main results in section [3] require also p > 2. Moreover, in the set of Assumptions I3.ll 

uj < =^> A > up. 

Indeed, from [8.b] follows p > 1 which implies {A : u < A < pu} = 0, while from [8. a] 
follows p > 2 which implies {A : 2u < \ < pu} = 

Proof. In what follows we denote by C a positive constant not depending on t, x and u. 
Inequality (|4.7j) holds by means Holder's inequality. From (14. 7p follows 



|y(r)| v , <max{||S|| W /),e" w '*}e WT \x 



(4.10) 



\u(a)\uda 



< Ce u 



\ X \ V , +6( t , T ) q h\\L p x (t,T;U) 
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so that also H4.8[) is proven. To prove (14.91) we need to estimate the right hand side in 
(4.11) f e-^\y{r)\ vl < j(e~ Xt - e' Xs )\x\ vl + C\\u\\ Ll{t>s;U) J° e -^6(t, t)Ut 
Indeed, in case A > up, one derives 



(4.12) 



while similarly in case A < up, one obtains 

(we recall that in such case A — to > in view of Remark 14. 6p . Hence, when A ^ up, we 
have 

(4.13) [ S e-^0(t,r)Ur < C|MU (t , s;C/) e^^, 



for all s. In the case A = up, on the other hand, there exists 5 > such that X — u — 5 > 0, 
and consequently T$ >t such that 



Then 



(4.14) 



e~ 6T \r -tU < 1, Vr > 7>. 



e- {x - w)T 9(t,T)«dr < J e- {x - w)T 9(t,T)«dT 

i>T s p+oo 

<\T s -t\< e~ ( - x -^ T dr+ e~^- w ~^ T dr 

Jt Jt s 

< 1 \ Ts _ t|i( e -( A -)* - e -^ Ts ) + — - — , e -(^-m 

A — u X — u — 5 

< C 



for a suitable constant C . Then applying (14.141) and (I4.13P to (14. lip one derives (14.91) . I 

Lemma 4.7. Let Assumptions \3. 1\ be satisfied. Let e £ [0, 1] be fixed, u £ £ L p x (t, s; U) be 
any e-optimal control at (t, x), with horizon s for the functional J s (t, x, ■) defined in (\3.2R) . 
Then, for a suitable positive constant K, independent of t, s and x, we have: 

(i) \\u e \\ L p ( tiS .u} < K{1 + |a?| 2 ,), when Assumptions \3. 1][ 8. a] holds; 

(ii) ||w e ||L p (t,s;!7) — K(l + \x\ v ,), when Assumptions \3. 1\[ 8. b] holds. 

Proof. Let u £ dom(ho), and u{t) = u. Let also {u £ ,y £ ) be e-optimal at (t, x). Then 
(4.15) J s (t,x,u £ ) - e < J s (t,x,u). 
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On one hand, from the convexity of go and <fio, and from (14.91) . there exists some positive 
constant C such that 



J s {t,x,u e )> J e- A -(aK(r)|^ + 6)dr-C o y e~ XT (l + \y £ {r)\ v ,)dr+ 

-C e~ Xs (l + \y £ (s)\ vl ) 
(416) ^ i \y,K)\ v ,) 

^ a \KW P Ll(t,s;U) + - C d X \v> + \K\\Ll(t,s;U) + 1) + 

-C e- Xs - CCoe-( A -^(|a;| + || M£ || L P (M;C/) 0(t, S )^) 

where in the last estimate we applied the assumptions on h , and estimates (14. 8 p (14.91) . 
Since e~^ x ~^ s 9(t, s)« is bounded for all s, the latter implies 

(4-17) J s (t,X,U £ ) > a\\u e \\ P L P^ s . u} - 7l|KIU*(t, S ;C/) ~ ^ly, +73 

for a suitable choice of the constants 71,72,73- On the other hand, also J s (t,x,u) can be 
estimated by means of either Assumptions I3.1| 8.a] or [8.b]. Indeed, we derive that the 
trajectory y{r) = y(t,x,u) satisfies 

\y(r)\v < K ie ^(l + |x|vO, 

where K x = e^l V l^l^- 1 ). Then, if [8. a] holds, \y{f)\v> < 2Kfe 2ujT {l + \x\^,), so 
that 

J s {t, x, u) < (\h (u)\ + MsJA- 1 + 2\g \ B2 K 2 (l + \x\ 2 v ,)(X - 2u)~ 1 + 

(4.18) + |0 o | B2 e- As (l + 2K 2 e 2 ^{l + \x\ 2 v ,)) 

< 74(1 + |j:|^) 

for a suitable constant 74. Hence, by means of (I4.15p . (14.171) and (14.181) . we obtain 

ll^ll^(t lS ;C/)( a ll^ll^( M ;C7) ~ 7l) < (272 + 7 4 )(1 + \x\ 2 ) ~ 7 3 + £ 

which imply (i). If instead [8.b] holds, then by a similar reasoning one derives 

(4.19) J s (t,x,u) <75(1 + Mv) 
for a suitable constant 75, and then (ii). | 

Lemma 4.8. Let Assumptions \3. 1\ be satisfied. If Assumptions [Q1 hold with [8. a]), then 
\I> satisfies 

3C>0 : \9{t,x)\ < C(l + \x\ 2 v> ), V(t,x) <E [0,+oo[xV. 
If Assumptions HOI hold with [8.b]), then ^ satisfies 

3C>0 : \%(t,x)\ < C(l + \x\ v >), V(t,x) e [0,+oo[xV. 
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Proof. Let e > be fixed and u £ be an admissible control, with y £ (r) = y(r; 0,x,u E ) the 
associated trajectory, such that 

*{t,x)> [ e- XT [g {y £ (T)) + h {u £ (r))]dT + e- xt Mye{t))-e. 
Jo 

Hence from the convexity of g and h , for a suitable positive constant 7, and by applying 
(14. 91) we derive 



V(t,x) > -7 / e 
Jo 

7 



-At 



1 + \y E {r)\v> + \u e (r)\u 



dr — e 



-~\~ lC [\ X \v> + W U z\\Ll(0,t;U) + 1 ]-l\ 1 - e Xt \ q ^ v \K\\Ll(0,t;U) ~£ 



so that by means of Lemma 14.71 

-*(t,z) <C(l + |x^,) 
when Assumptions I3.1| 8.a] holds, and 

< C(l + |z|y/), 



when Assumptions I3.1| 8.b] holds, for a suitable choice of the constant C . The missing 
inequality derives from 

W(t,x) < J t (0,x,u) 



with u(t) = u G dom(ho), when we apply (14.181) when [8. a] holds, or (14.191) when [8.b] 
holds. ■ 



Lemma 4.9. Let Assumptions \3. 1\ hold, and A > co>max{2,g}. Then 
(i) sup[$ x (t)] L < +00; 



t>o 



(U) SUp 0)|y < +OO. 

t>0 



Proof. We use estimate ( 12. 8p with g(s,x) = e go(x), and <£>(x) = e XT (j)o(x) to derive 



so that 



for all t > 0. 



A — 



[^(t)]L = e A(T - t) [0,W]L 

-(A-2u,)t U / 



A — uj 



W ]l 
oJl ^A^? 



< m L + 
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Next we prove (ii). Let hbe a real number \h\ < 1 and z 6 V such that \z\y < 1- Let 
u £ is e-optimal at (0, 0) with horizon t, 2/o,e( s ) := y( s ', 0, 0, u e ) and yh,s{s) '■= y( s ', 0, foz, w £ ), 
then by means of (14.81) one has 



\yo,e(t)\ v , < WBWuuy^eio, t^\\u E \\ LP{0AU) 



so that 

(4.20) 
#(t, /wr) - *(t,0) 
ft 



< 



< I e- Xs {g' (y Qi£ (s)),e" s \z\ v ,) v ,ds + e xt {<t>' Q {y Q , E {t)),e^\z\ ,) v , -e 



At / ±i 



< X -^ S (1 + \y ,M\v) ds + e-^\<P',\ Bl {l + M*)|vO 





KW [ 




Jo 







— e 



+ I0olsi( e " 



-(A-u/)t + e -(A 



- 2 ^(o,t)*; 



— £ 



Recalling Lemma 14.71 using that A > qu and reasoning like in ( 14. 13ft and ( 14.141) . one 
obtains that the following quantities 



are bounded by a constant (independent of t), by passing to limits as h — > in the 
preceding inequality one derives 

sup{^ x (t, 0), z) < +oo. 
t>o 

On the other hand, by a similar reasoning, and with Uh, £ ^-optimal at (0, hz) with horizon 
t, there exists a positive constant r) such that 



(4.21) 
so that 



tf(t,0) - *(t, hz) 
h 



< T)\z\v' — £ 



swp{9 x (t,0),-z) < +oo, 

t>0 



and the proof is complete. 
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4.2 Proof of Theorem 15151 

We divide the long proof into several steps. Let x G V be fixed, \x\ , < r, and let 
< t x < t 2 . 

Claim 1: Let e > be fixed and let u £ G -£^(0, t; U) be e-optimal at starting point (0, x) 
with horizon t, and y e (s) := y(s; 0, x,u E ) be the associated trajectory. Then there exists a 
bounded continuous function p, depending only from r, with lim^ +00 p{t) = 0, and such 
that: 

e ~ Xt \Ve{t)\ 2 , < pit), if Assumptions [X71 [8. a] are satisfied; 
e~ xt \y £ (t)\ v , < pit), if Assumptions l3J\ [8. b] are satisfied. 
Indeed, applying (14. 7p and Lemma H~71 we derive 



(4.22) e- xt \y £ {t)\ v , < K^-^ [l + 0(O,t)« 
with K r a suitable constant. Hence for a (possibly different) constant K r > 0, we have 

(4.23) e- xt \y £ {t)\l, < K r e~^\l + 6{^t)\ + 6{^t)l). 

By proceeding as in the proof of Lemma 14.51 one sees that the following functions are 
infinitesimal as t goes to +oo 

e -(*-"O*0(o,t)«, e - (A - 2w) ^(o,t)«, 

so that what is left to show is 

e -(A-!Mt0(o,*)f, *->0. 

The property is straightforward in the case A = up, as 

e -(A-:Mt0( Oji )f = e -^-Mt\ t \\ i 

while for A < up one has 
Finally, if A > c<jp, 

e -(A-2^ (M) § < Ce _(A-2.)t e 2(A^) t = 

and Claim 1 is proved. 

Claim 2: lim ^(h,x) -^(t 2 ,x) <0 

Let e > be arbitrarily fixed, let w e G 1^(0, £2! ^0 be such that 

$(t 2 ,x) > J t2 (0,x,u £ ) - e 
and let y e (r) := y(r; 0, x, u e ). Then 

*{t 2 ,x) > / e- As [(7o(2/ £ (s)) + /i (M e (s))]cis + e- A * 1 J 42 ^ 1 (0,y £ (t 1 ),u £ (- + t 1 ))-£ 



(4.24) 

> - e-^fafefa)) + e~ xt ^{t 2 - t u y^)) - e 
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where we used (14. 6p . Consequently, when [8. a] holds 

(4.25) *{t u x) - *{t 2 ,x) < Ce- Xh (l + \y £ ( tl )\ 2 v/ ) + e 
while for [8.b] 

(4.26) ^(h,x) - *(t 2 ,x) < Ce- A4l (l + \y E (ti)\ v ,) + e 

for a suitable constant C, and the last implies Claim 2 as a consequence of Claim 1, and 
Lemma 14.81 

Claim 3: lim ^(t 2 ,x) - V(ti,x) < 

If we choose e > and v £ £ L p x (0,ti, U) so that 

> Jti(0,x,i; e ) -e, 

and set y e (s) := y(s, 0,x, w e ), then by the DDP contained in (14.51) we obtain 

(4.27) *(t 2 , x) - *(t 1? x) < e- A '^(t 2 - h, y^h)) - e- Ml MVe(ti)) + e 

which leads as before to the conclusion. Since the constants involved in the estimates 
are uniform in x, for x varying in a bounded subset of V, we derive the convergence 
x) — > ^oo(x), as t — ► +oo is uniform on bounded subsets of V. 
Next we discuss the convergence of gradients. 

Claim 4: ^oo is Frechet differentiable, with differential and, for every fixed x £ V 

^(t, x) — > ^'^(x), weakly in V, as £ — ► +oo. 

Let x be fixed in V, h a real parameter, with h £ [—1, 1], y in V with \y\v' < 1, and 
x, y) = £t(h) :=*(t,x + hy). Then 

&(h) := (* x (t,x + hy),y). 

Note that, since £t(/i) — > (,oo(h) = ^^(x+hy) as t — > +oo, if we show that ££(7i) converges 
uniformly in [—1,1] to some function as t — > +oo (or along a subsequence), then such 
function is ^{h). We do so by means of Ascoli-Arzela Theorem. We have 

\^-e t (k)\<\y\UMt))]L\h-kk 

which implies, by Lemma l431 (i). that the family {£,' t }t>o is equicontinuous (more precisely, 
equilipschitzean) . Moreover 

(42g) Wt(h)\ < \y\v>\**(t,x + hy)\ v 

< \y\v>([*S)] L \x + hy\v> + \*x(t,0)\v) 

from which follows, by means of Lemma 14.91 (ii), that {Ct}t>o is uniformly bounded. 
Consequently, is Gateaux differentiable. Indeed, there exists the following limit 

Voo(x + hy) -*oo(x) , 
lim 7 = ^oo(O) =: (V*oo(x), y) v> , 

18 



where V^/oo indicates the Gateaux differential of \IZoo. In particular, what we prove implies 
also 

ty x (t,x) — > V\l / 00 (a;) weakly in V as t — > +00. 
Finally we show that V^oo is continuous. It suffices to pass to limits as t goes to 00 in 



\e t (0;x,y)-e t (0;z,y)\ < \y\ v ,\* x (t,x)-*(t,z) 



v 



( ^ 4 ' 29 ' ) < \y\ysup[^/ x (t)} L \x - z\ 



t>0 

Hence is Frechet differentiable with Frechet differential = V^oq. The proof that 
^/oo is convex and in C\ ip {V') is trivial by means of Lemma [4.91 

4.3 Proof of Theorem [372 (i) 

First of all we show that, for any fixed t, x and u G -^(0, +00) we have 
(4.30) 3 lim J t (0, x, u) = Joo(0, x, u). 

t—>+oo 

We separately show that, if y(s) = y(s; 0, x, u), then 
(4.31) 

lim \e~ xt (f) (y(t))\ = 0, and J 00 (t,x,u)= lim / e - AT [^ (2/(r)) + h {u(r)))dT. 

t— >+oo 1 1 5— >+oo 







Indeed 

\y(t)\ v , < Ke ut (i + e(o, ty< 

where K = \x\ v , V ||u||i*(o,+ o i i7)- Hence 



(4-32) e- xt \y(t)t<p(t), e- xt \y(t)\ , < p(t) 



where p(£) denotes some positive function with lim^ +00 p(t) = (obtained as in proof 
of Claim 1 of Theorem 13.61) . These inequalities combined with Assumptions 13. ll [8. a] or 
[8.b] - recall that 4>o and go are either sublinear or subquadratic - give the first equality 
in (I4.3ip . and by means of dominated convergence also 



r-t r+00 

/ e~ Xs g {y{s))ds -> / e~ Xs g (y{s))ds, as t -> +00. 
Jo Jo 



To complete the proof of (14.31 j) we just observe 

Asl /„./„\\j„ . / — As j 



e s h (u(s))ds — > / e s h (u(s))ds, as t — > +00 
io 

by monotone convergence, for /io is bounded from below. 

Then ( 14.301) is proved. As a consequence, by passing to limits as t tends to +00 in 

V(t,x) <J t (Q,x,u), Vw G L p x (0, +00; Z7) 
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we obtain 

^ooOr) < Joo(0,x,m), Vm € L p x (0,+oo;U) 

which implies 

We now need to show that the reverse inequality holds. Let e > be fixed. From (14.151) 
and (I4.18P we derive that if u* is e-optimal at (0, x) with horizon +oo 

(4.33) J^O^O <C(l + \x\l)+e 



with C a suitable constant. Hence, we set 
(4.34) u t (s) -- 



Ui(s) s G [0,t] 
u 2 (s) sG]t,+oo[ 



where u% G L p x (0,t;U) is e-optimal for J t at (0,x), and u 2 G L p x (t, +oo; U) is ^-optimal 
for at (0, yt{t))i with yt(s) := y(s; 0, x, u t ), and we derive by means of (14 .6p and (14.3c 
the following chain of inequalities 



Zoo(0,x) < Joo(0, X, u t ) 

= J t {0, x, u x ) + e~ xt [./^(O, y t {t),u 2 ) - Mvt(t))] 
< *(t,x) + Ce- X \l + \y t {t)\ v , + \y t (t)\ 2 v ,) + 2^ 
=: $(t,x)+p(t)+2£ 



and with C some suitable constant (possibly different from the one mentioned above). 
Note that p(t) — > 0, as t —>■ +oo as one derives from Claim 1 in the proof of Theorem 13 .61 
Hence, by passing to limits as t goes to +oo, we derive 

Zoc(0,x) < ^oo(x), 

and the proof is complete. 

4.4 Proof of Theorem 13.71 (ii) 

To prove the theorem we make use of the Dynamic Programming Principle (DPP from 
now on) contained in the following Lemma. 

Lemma 4.10. We have 

* O0 (x)= inf (/ e- Xs (g (y(s)) + h (u(s)))ds + (</(£))) , Vt > 0. 

Moreover, given any e > 0, if u e is such that 

Joo(0,x,m £ ) < *oo(a;) +e 

then also 

e- Xs (g (y(s)) + h (u(s)))ds + e- xt * 0O (y(t)) < ^oo(x) + e 



20 



The proof of this lemma is standard and we omit it. 

We prove then Theorem 13.71 (ii). Let t > be fixed. Let also u(s) = u G dom(h ), and 
let y(s) := y(s; 0, x, u). Then the DPP implies 

(4 36) e-^.(gw)-«.(x) > i r e -, Mm + Ma)]lfa . 

fc I Jo 

Since 

y(t) - y(0) e A *x - x 1 ''' 



1 f 

+ - e A(t ~ s) Buds -> Ax + SiZ, as t -> 0, 



and s i— > (7o(y('5))e _As G C(0, t; £/), then we may pass to limits in (I4.36P and obtain 

-A* 00 (x) + (^Jx), Ax) + (^(x), J3u) + ho(u) + g (x) > 0, 

and take the infimum of both sides as u G dom(ho) and derive 

-A\Mx) + «,(*), Ax) - ^(-^^(x)) + g{x) > 0. 

Next we prove the reverse inequality. Let e > and t G [0, 1] arbitrarily fixed, and let u e 
be an et- optimal control at (0, x) with horizon +oo, and y e be the associated trajectory. 
Then by the DPP 

e- Xt ^oo(ys(t)) - ^oo(x) + [ e- Xs [g (y £ (s)) + h (u £ (s))}ds < et, Vt G [0, 1]. 

Jo 

Since \IZoo is convex and differentiable, the preceding implies 
e^^'ooW,^^)-^) 1 "'^ 



(4-37) 1 r* 

+ 

Now we show that 



7 / e- Xs [g (y £ (s)) + h (u e (s))]ds < e, Vt G [0, 1]. 
1 Jo 



i r* 

(4.38) -/ e- Xs go(y £ (s))ds = go(x) + p(t) 

where by pit), we denote some real function not depending on u e such that p{t) — > as 
£ — > 0. Indeed the assumptions on go imply that 



\go{x) - go(y)\ < C{\x\ \y\ v ,)\x - y\ 



V 



with 

C(a,f3) := Ml + |^(0)|) (l+aV^S). 
Moreover by Lemma 14.71 and by (14. 7p we derive 



(4.39) \y e (s) - x\ v , < \e sA x - x\ yl + C(l + \x\ 2 v ,)e" B 8(0, s 
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for some constant C (independent of u £ and x) and with 8 the function defined in Lemma 
14. 5[ which has as a consequence 

sup \y £ (s)\ v , < K(x) < +00, 

se[o,i] 

with K(x) not depending on u £ . Hence for t G [0, 1] 
(4.40) 

7/ \e- Xs g {ye{s))ds - g {x)\ds < - / e- Xs \g (y E (s)) - g {x)\ds + p(t) 
1 Jo 1 Jo 



<C(\x\ v „K(x)) 
+ P{t), 



1 rt 1 rt 



C{l + \x\ 2 vl )- I e' {x - u})s 8(0,s)^ds + - J e~ Xs \e sA x - x\ yl ds 



which implies (14.381) by definition of 8(0, s). 
Observe now that, clS X G D(A), then 

Ve(t)-X _ , , . . , 1 



1 r* 

Ax + p(t) + - / e {t - s)A Bu £ (s)ds. 
t Jo 



t 

Then, the last and (14.381) imply that (I4.37P can be written as 

(^(x), Ax) - A^oo(x) + #(x)+ 
(4.41) + - r e -^[( e -^-) J B* e (*- s ) A ^' 00 (x) )Ue ( S )) + / i oK( S ))]^ < e + p(t), 

c JO 
Vt G [0,1]. 

We then get that 

1 y* 



(4.42) 



> 



e" As 



[( e -A(*-) jB * e (*-)A^/ oo(:c ) )Ue(s) ) + /io(u£ ( s ) ) ] fis > 

- / e- As sup [(-e- A( *- s) J B*e ( *- s) ^*' 0O (x),«) - fco(«)]ds 

- fe- Xs h* (-e- x ^B*e^ A *y oo (x)) ds 
t Jo 



= -K(-B*V 00 (x))+p(t), 
for /iq G C^j (Z7) by assumption. Hence 

(^(z), Ac) - A^x) + <7(x) - ^(--B**^)) < e + p(f), Vt G [0,1], 
which implies the thesis by passing to limits as t — >• 0. 

4.5 Verification Theorem 

We state and prove the following verification theorem: 
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Theorem 4.11. Let Assumptions \3.1\ hold. Let t > and x G V be fixed. Then 
(4.43) 

e-^^x) = J^x^-J e-*[ht(-B**' x {y(s)))+{B**' x {y(s)) | u(s)) u+ h (u(s))}]ds. 

As a consequence, an admissible pair (u, y) at (t, x) is optimal if and only if 

sup {(«| - B*Vj3,{s))) v - h (u)} = (u(s)\ - B^yis))^ - h (u(s)) 

for a.e. s > 0, which is equivalent to 

u( s ) = Wi-irvMs)) 

for a.e. s > 0. 

Proof. Let first x G D(A), t > be fixed. Let u be any admissible control at (t,x) such 
that JT(t,x,u) < +oo for every T > 0. (Note that an admissible control violating this 
condition cannot be optimal for the infinite horizon problem, as Jx{t,x,u) = +oo for 
some T > implies Joo(t, x,u) = +oo.) Let y be the associated trajectory. Then for a.e. 
s G [t, +oo[ we may differentiate e~ Xs ^ 00 (y(s)) as function of s to obtain 

(4.44) 

^-e-^Ms)) = -Xe-^Ms)) + e-^^Ma)), Ay (a) + Bu(s)) 
as 

= W-B'VMs))) - g (y(s)) + (5**^00) \ u(s))u + e- Xs h (u(s)) - e- Xs h (u(s)) 

where we used the fact that solves the stationary HJB equation, and we added and 
subtracted the term e~ Xs ho(u(s)). Integrating such equation on [t,T] we have 

e- AT ^ 00 (y(T))-e- A ^ 00 (x) = 
(4.45) = £ e- Xs [K(-B^> 00 (y(s))) + (B'V'Ms)) \ u(s))u + h (u(s))}]ds+ 

- J T (t,x,u) + e- XT My(T)). 

Such relation holds for all admissible controls. If now we show that, for any fixed admis- 
sible control u, is 

e- AT ^ 00 (y(T)) -> 0, and e- XT My(T)) 0, as T -> +oo, 

then we may pass to limits in (I4.45P and derive (j4.43|) . Indeed, using the fact that 4>q and 
^oo are either sublinear or subquadratic, we observe that, by (14.321) and (??) 

e- AT |*oo(2/(T)) + Mv(T))\ < CKp{T), 

where K and p are the same there considered, and C a suitable positive constant. Then 
04.43p is proved. It also implies that ^^{x) < Joo(0, x,u) and that the equality holds if 
and only if 

oo 

e~ Xs [M-irVMs))) + {B'VMs)) I u(s))u + h (u(s))}] ds = 
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which means, by the positivity of the integrand that 



h* (-B**Uy(s))) = (-B^Uvis)) I u(s))u - h (u(s)) 

for almost every s > t. The claim easily follows from the definition of h^. The claim 
for generic x G V follows by approximating it by a sequence of elements of D(A) and 
observing that the relation (I4.43P make sense also for x G V . ■ 



4.6 Proof of Theorem ET51 

We first observe that the closed loop equation has a unique solution y* since the feedback 
map defined as 

G(x) = (hZYi-B**^)) 

is Lipschitz continuous, as one may show by a standard fixed point argument. 
Next we prove that the control 

«*(«) = (Kn-B^y^s)) 

is admissible, i.e. it belongs to L p x (t, +00; U). To do so, we first observe that the relation 
(I4.45P holds true for any control u G Lj oc (t, +00; U) such that Jr(t, x, u) < +00 for every 
T > 0. Since by definition we have 

h (u*(s)) = (-B^' 00 (y*(s))\u*(s)) u - K {-BrVJifis))) 

then also Jx{t,x,u*) < +00 for every T > 0. So we get that u* satisfies 

(4.46) J T (t, x, u*) - e- XT My*(T)) = - e~ XT ^ ^{T)). 

By means of H4.8[) and proceeding as in the proof of Lemma 14.51 one derives 

J e ~ XS \V*{ s )\ vl ds < 71 + 72||«*||^(t,T;t/) 

where 71,72 are suitable constants (depending on x, t), so that 

A: = J T (t,x,u*)-e- XT My*(T)) 

>[ e- Xs [a\u*(s)\u + b]ds-C [ e~ Xs [l + \y*(s)\ v/ ]ds 
Jt Jt 



(4.47) 



> a\\u 



* hp 



b-C 



Ll(t,T;U) + — 71 -72||M ||i^,T;[/) 



> a\\u ||^ (t!T;C/) " J2\\U \\ L {(t,T;U) ~ 73 

for a suitable constant 73. On the other hand, since by means of (14.81) . and proceeding 
again like in the proof of Lemma I4.5[ we have 

e" AT (l + \y*(T)\ v ,) < 74 + i 5 \W*\\ L l(t,T;u) 
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where 74,75 are suitable constants (depending on x, t), so that, using Lemma H~8l 

(4 4g) B -^e-^^x) - e- Ar v& 00 (y*(T)) 

<e- A **oo(^-C(74 + 75||w*||L-( tl T ; t/)) 
Hence combining this inequality with (I4.47P and (14.481) we obtain 

a \\ U *W P Ll(t,T;U) ~ 7\\u*\\ L l(t,T;U) < V 

for suitable positive constants 7 and 77 independent of T. Then we may pass to limits as 
T goes to +00 and derive 

which implies that u* G L p x (t, +00; U), and is hence admissible. 

Then by the above Theorem 14.111 we get that the couple (u*,y*) is optimal. The 
uniqueness follows by the uniqueness of the solution of the closed loop equation. 



4.7 Proof of Thorem 13.71 (Hi) 



Let t/> G C 1 be any other function that satisfies the stationary HJB equation (13 .4p in 
classical sense as from Definition 13.31 Then we have, arguing exactly as in the proof 
of Theorem 14.111 that ip satisfies the fundamental relation (14.431) in place of ^oo- This 
implies, setting t = 0, that i/j(x) < Joo(0, x,u) and consequently 

lj)(x) < *oo(x). 

Moreover if we find an admissible control u at x such that 

(h* o y(-B*iP'(y(s))) = (-BV(y(»)) | u(s))u - h (u(s)) 

then ip(x) = Joo(0,x,u), u is optimal, and then ip(x) = ^00(0;). Such a control exists as 
one may derive arguing as in the proof of the Theorem 13.81 

5 The economic example of optimal investment with 
vintage capital 

We here describe our motivating example: the infinite horizon problem of optimal invest- 
ment with vintage capital, in the setting introduced by Barucci and Gozzi [H][I2], and 
later reprised and generalized by Feichtinger et al. [281 [221 EU] , and by Faggian [2] 
The capital accumulation is described by the following system 

^T 1 + T + Mr, s) = Ul (r, s), (r, s) e)t, +oo[x]0, s] 
('>■!) { y(r,0) = uo(r), r e]t, +00 [ 

y(t,s) = x(s), s G [0,s] 
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with t > the initial time, s G [0, +00] the maximal allowed age, and r G [0,T[ with 
horizon T = +00. The unknown y(r, s) represents the amount of capital goods of age s 
accumulated at time r, the initial datum is a function x G L 2 (0, s), \i > is a depreciation 
factor. Moreover, uq : [t, +00 [— > R is the investment in new capital goods (uq is the 
boundary control) while U\ : [t, +oo[x[0, s] — > R is the investment at time r in capital 
goods of age s (hence, the distributed control). Investments are jointly referred to as the 
control u = (u , Ui). 

Besides, we consider the firm profits represented by the functional 

/+00 
e- Xr [R{Q{r)) -c{u{r))]dr 

where, for some given measurable coefficient a, we have that 

Q(t) = / a(s)y(r,s)ds 
Jo 

is the output rate (linear in y{r)) R is a concave revenue from Q(t) (i.e., from y(r)). 
Moreover we have 

c(tto(r),tti(r)) = / ci(s,tti(r,s))ds + co(ifo(r)), 
Jo 

with C\ indicating the investment cost rate for technologies of age s, Cq the investment 
cost in new technologies, including adjustment-innovation, Cq, C\ convex in the control 
variables. 

The entrepreneur's problem is that of maximizing I(t, x; uq, U\) over all state-control 
pairs {y, (uo,Ui)} which are solutions in a suitable sense of equation (|5.ip . Such problems 
are known as vintage capital problems, for the capital goods depend jointly on time r and 
on age s, which is equivalent to their dependence from time and vintage r — s. 

When rephrased in an infinite dimensional setting, with H := L 2 (0, s) as state space, 
the state equation (15.1 j) can be reformulated as a linear control system with an unbounded 
control operator, that is 



(5.2) 



y'{r) = A y(T) + Bu(r), rG]t,+oc[; 
y(t) = x, 



where y : [t, +oo[— > H, x G H, A : D(A ) C H — > H is the infinitesimal generator of 
a strongly continuous semigroup {e Aot } t >o on H with domain D(A ) — {/ G if 1 (0,s) : 
/(0) = 0} and defined as A f(s) = —f'(s) — fJ-f(s), the control space is U = R x H, the 
control function is a couple u = (uq, U\) : [t, +oo[— > R x H, and the control operator is 
given by Bu = B(uq, U\) — U\ + u 5q, for all (uq, u±) G R x H, 6q being the Dirac delta at 
the point 0. Note that, although B £ L(U, H), is B G L(U, D(A* )'). The reader can find 
in [TTJthe (simple) proof of the following theorem, which we will exploit in a short while. 
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Theorem 5.1. Given any initial datum x G H and control u G L p x (t,+oo;U) the mild 
solution of the equation ( f5.^|) 

y (s) = e {s ~ t)A x + J' e {s - T)A Bu(T)dr 

belongs to C([t, +00) ; H) . 



Following Remark 12. 2\ we then set 

V = D(A* ) = {feH 1 (Ors):f(s) = 0} 
and V = D(Aq)'. Regarding the target functional, we set 

Joo(t,x;M) := -I(t,x;u ,Ui), 

with: 

go :V'^R, g (x) = -R((a,x))_, 
h :U-^R, h {u) = c (u ) + £ ci(s, %(s))ds. 

Remark 5.2. As announced in Remark 13.41 here the extension of the datum g$ to V is 
straightforward, as long as we assume that a G V and replace scalar product in H with 
the duality in V, V. 

Note further that uj = 0, A > (the type of the semigroup is negative and equal to 
-//). ■ 



As the problem now fits into our abstract setting, the main results of the previous 
sections apply to the economic problem when data R, c , C\ satisfy Assumption 13. 1( 8. a] 
or [8.b]. In particular, such thing happens in the following two interesting cases: 

• If we assume, for instance, that R is a concave, C , sublinear function (for example 
one could take R quadratic in a bounded set and then take its linear continuation, see 
e.g. [SUED]), and cq, c\ quadratic functions of the control variable, then Assumption 
EH8.b] holds. 

• Assumption I3.1| 8.a] is instead satisfied when R is, for instance, quadratic - as it 
occurs in some other meaningful economic problems - and cq, c\ are equal to +00 
outside some compact interval, and equal to any convex l.s.c. function otherwise. 
Such case corresponds to that of constrained controls (controls that violate the 
constrain yield infinite costs). 

In these cases, Theorems 13. 6} 13.71 13.81 hold true. In particular Theorem 13.81 states the 
existence of a unique optimal pair (u*,y*) for any initial datum x G V. Note that in 
general the optimal trajectory y* lives in V . However, since the economic problem makes 
sense in H, we would now like to infer that whenever x (the initial age distribution of 
capital) lies in H, then the whole optimal trajectory lives in H. Indeed, this is guaranteed 
by Theorem 15. 11 
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All these results allow to perform the analysis of the behavior of the optimal pairs and 
to study phenomena such as the diffusion of new technologies (see e.g. [HI [12]) and the 
anticipation effects (see e.g. [281 [30]). With respect to the results in [HI [12], here also the 
case of nonlinear R (which is particularly interesting from the economic point of view, as 
it takes into account the case of large investors) is considered. With respect to the results 
in [211 [30], here the existence of optimal feedbacks yields a tool to study more deeply the 
long run behavior of the trajectories, like the presence of long run equilibrium points and 
their properties. 
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